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ABSTRACT 


This  report  describes  a  large-scale  computerized  system  for  Selective 
Dissemination  of  Information  (SDl)  developed  over  the  past  five  years  at  the 
U.S.  Army  Electronics  Command  to  serve  its  technical  personnel.  The  system, 
which  uses  as  its  document  base  the  current  accessions  of  the  Defense  Docu¬ 
mentation  Center,  was  developed  in  three  phases :  an  18-month  semiautomated 
test  with  50  subscribers,  a  fully  automated  system  with  about  500  subscribers, 
and  a  proposed  standard  system  for  Army  use.  System  descriptions  include 
profiling  techniques,  keyword  matching  strategies,  processing  procedures, 
flow  charts,  and  samples  of  inputs  and  outputs .  System  effectiveness  is 
evaluated  in  terms  of  document  ordering,  relevance  of  citations,  and  the 
impact  the  system  has  had  on  the  R&D  program  as  revealed  by  responses  to 
que s ti onnaire s . 


ii 


CONTENTS 

Page 

INTRODUCTION  1 

PHASE  I  SYSTEM  ( SEMIAUTOMATED  PILOT  TEST)  2 

PHASE  II  SYSTEM  (FULLY  COMPUTERIZED  SERVICE)  5 

PHASE  III  (DEVELOPMENT  OF  STANDARD  ARMY  SYSTEM)  19 

FUTURE  PLANS  23 

CONCLUSIONS  AND  RECOMMENDATIONS  25 

REFERENCES  25 

TABLES 

I.  Typical  Semimonthly  Run  in  the  Phase  I  Pilot  Test  3 

II.  Document  Ordering  for  Latest  15  Runs  18 

III.  Cumulated  Data  for  Latest  15  Runs  18 

FIGURES 

1 .  "Selected  Abstracts"  Booklet  (Phase  I  System)  4 

2.  Simplified  System  Flow  Chart  (Phase  II  System)  6 

3.  Pre-addressed  Cover  Sheet  for  "Selected  Abstracts"  Booklet  7 

4.  Abstract  Printout  8 

5.  Hits  for  Each  Subscriber  by  Keyword  9 

6.  Number  of  Abstracts  Referred  to  Each  Subscriber  10 

7.  Current  Profiles  11 

8.  Hits  Alphabetically  by  Keyword  12 

9.  Users  for  Each  Keyword  13 

10.  Sample  Citation  15 

11  a. Flow  Chart  for  Phase  III  System,  Part  1  21 

11b. Flow  Chart  for  Phase  III  System,  Part  2  22 

12.  Sample  Microfiche  24 

APPENDIX 

RUN  DESCRIPTIONS  FOR  PHASE  II  PROGRAM  26 

FIGURES 

A-1  Tape  Conversion  28 

A-2  Selection  of  Citations  29 

A-3  Statistical  Reports  30 

A -4  File  Updating  and  Validation  31 


iii 


INTRODUCTION 


During  the  past  five  years,  the  U.S.  Army  Electronics  Command  developed 
and  tested  a  large-scale  computer  system  designed  to  notify  its  technical 
personnel  of  newly  published  literature  in  their  individual  fields  of 
interest.  This  Selective  Dissemination  of  Information  (SDI)  system,  now 
serving  more  than  500  subscribers ,  is  intended  to  keep  scientists,  engineers, 
and  project  managers  continuously  aware  of  the  latest  technical  developments 
that  might  affect  their  work. 

The  primary  product  of  the  ECCM  SDI  system  is  a  computer-produced  book¬ 
let  called  "Selected  Abstracts,"  issued  semimonthly  to  each  subscriber.  Each 
booklet  contains  a  special  selection  of  document  references  selected  by  the 
Burroughs-5500  computer  as  of  potential  interest  to  the  individual  subscriber. 

The  subscriber  indicates  his  interests  to  the  computer  by  preparing  a 
list  of  subject  keywords ,  called  his  "interest  profile."  These  interest 
statements ,  compiled  on  a  master  tape,  are  run  in  the  computer  semimonthly 
against  a  computer  tape  containing  abstracts  of  about  2,000  newly  published 
technical  documents ,  each  similarly  coded  with  subject  keywords .  By  comparing 
profile  keywords  and  abstract  keywords ,  the  computer  selects  those  abstracts 
that  are  within  each  subscriber's  specified  interest  areas. 

The  document  abstract  tape,  against  which  the  interest  profiles  are 
matched,  is  provided  cost-free  to  EC0M  by  the  Defense  Documentation  Center 
(DDC) .  The  tape  represents  the  most  complete  collection  of  newly  generated 
technical  reports  and  articles  resulting  from  the  multibillion  dollar 
Department  of  Defense  research,  development,  test  and  evaluation  program. 

A  typical  Selected  Abstracts  booklet  prepared  for  a  subscriber  semi¬ 
monthly  contains  about  16  document  citations,  from  which  the  average  subscriber 
orders  about  two  documents .  Copies  of  requested  documents  are  forwarded  to 
the  user  on  request,  usually  within  10  days. 

The  EC0M  SDI  system  was  developed  in  three  phases:  (l)  an  18-month 
semiautomated  test  with  about  50  subscribers ;  (2)  a  fully  automated  pilot 
system  with  subscribership  growing  from  50  to  about  500;  and  (3)  a  fully 
automated  standard  system  for  possible  Army-wide  adoption,  incorporating 
changes  and  improvements  found  advisable  from  pilot  test  experience . 

Among  the  benefits  to  be  expected  from  such  a  system  are:  reduced  time 
spent  by  scientists  in  literature  searching,  reduced  duplicative  experimen¬ 
tation,  greater  application  of  external  scientific  achievements ,  improved 
technical  proficiency,  greater  current  awareness ,  and  increased  productivity, 
all  of  which  contribute  to  reduced  cost  of  research  and  development  and  more 
rapid  development  of  new  military  equipment. 

Development  of  such  a  system  at  EC0M  was  undertaken  because  of  increased 
concern  over  what  is  commonly  termed  the  "information  explosion."  The  number 
of  reports  and  articles  being  published  by  the  worldwide  scientific  community 
has  reached  the  point  where  the  individual  scientist  often  cannot  keep  up 
with  the  literature,  even  in  his  narrow  field  of  specialization,  without 
devoting  a  prohibitive  amount  of  time  in  library  searching.  And  the  situa¬ 
tion  is  rapidly  worsening,  with  expectation  that  the  already  heavy  publication 
rate  will  double  in  10  years. 


It  is  generally  agreed  that  the  situation  requires  the  marshalling  of 
advanced  computer  technology  to  assist  the  researcher  in  filtering  this 
flood  of  new*  literature  for  data  that  can  be  applied  to  his  project.  The 
SDI  system  is  viewed  as  one  effective  method  to  achieve  this  end. 


According  to  a  recent  survey 


hers  are  96  installed  SDI  systems  in 


this  country  and  abroad ,  14  of  which' are  operated  by  U.  b.  Government  agen 


eies®  In  the  Army 
Biological  1 
Natick  Laboratories 


s©rs  are  three  other  systems;  one  at 


the  II.  S.  Army 


laboratory.  Fort  Be trick,  Frederick,  Md.j  one  at  the  U.  S»  Army 


Natick.  Mass.  ;  and  one  at  the  II. 


Mobility  Equip¬ 


ment  Research  and  Development  Center,,  Fort  Bel  voir ; 


Success  of  the  EGCM  system  is  largely  credited'  to  the  fact  that  it  is 
strongly  user-oriented ,  with  maximum  emphasis  on  service  and  convenience. 
Under  this  policy  ground  rule ,  subscribers  are  under  no  obxigation  to  order 
documents  announced  to  them  or  to  feed  back  information  to  the  system  once 
they  are  enrolled.  Xt  is  a  barometer  of  system  acceptance  that  despite  this 
fact,  document  ordering  is  heavy,  incidence  of  subscriber  dropout  is 
low,  and  response  to  questionnaires  is  heavy  end  favorable, 

PHASE  1  SYSTEM  (SEMXATJTOMATSD  PILOT  TEST) 


In  June 

'was  a  ready-made  opportunity  to  exploit  ih~-  csnT 


i3s  BOOM  decided  that  within  the  Department  of  Defense  there 
exploit  tht  SDI  concept,  using  the  current 
accessions  listing  provided  by  the  Defense  Documentation  Center  (BBC)  in  its 
Technical  Abstract  Bulletin  (TAB) ,  This  ssrsisonthly  publication  includes 
abstracts  of  the  latest  technical  reports  and  articles  in  all  fields  of 
science  and  technology  resulting  from  the  nationwide  Defense  HDTE  effort,  and 
contains  a  valuable  fund  of ‘ Information  that  had  been  largely  untapped  by 
ECOM  technical  personnel . 


In  the  TAB,  BBC  assigns  to  each  document  a  cluster  of  keywords  describ¬ 
ing  the  document’ s,,  content  from  a  list  of  keywords  (descriptors)  published  in 
the  DBG  thesaurus.2  ®DC  provides,  along  with  the  TAB,  a  complete  cross- 
index  to  the  primary  (starred)  keywords  for  each  document  so  that  all  docu¬ 
ments  with  a  certain  keyword  in  a  primary  capacity  can  be  readily  found. 


Thus,  to  provide  a  orimi tiv®  test-  SDI  service  at  ECOM,  all  that  was 
necessary  was  to  obtain  from  each  subscriber  a  list  of  terms  from  the  BBC 
thesaurus  that  described  his  interests  and,  tsrough  clerical  procedures,  to 
assemble  into  booklets  the  citations  that  had  those  terns  as  starred  key¬ 
words. 

Subscribershin 

A  preliminary  test  in  July  1963  with  two  subscribers  was  most  promising. 
Two  months  later,  ECOM  began  an  18-month  pilot  study  with  a  steadily  increas¬ 
ing  number  of  subscribers,  beginning  with  16  and  reaching  its  peak  at  56. 


Profiling 

Profiles  were  prepared  initially  by  interviewing  prospective  subscribers, 
but  it  was  found  after  a  few  months  that  subscribers  could  effectively  select 
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th®ir  own  list  of  keywords  from  the  DDC  thesaurus.  The  interview  method  was 
therefore  abandoned  in  favor  of  self-profiling,  with  professional  assistance 
provided  only  on  request.  The  self-profiling  system  was  further  refined  by 
preparation  of  a  User  Profiling  Handbook, 3  which  was  employed  exclusively 
during  the  last  few  months  of  the  Phase  I  test  and  through  the  entire  Phase  II 
period  to  date. 

Selection 

As  the  number  of  subscribers  grew,  it  became  evident  that  processing 
would  be  faster  and  more  economical  with  the  aid  of  punch  card  equipment, 
thus  the  following  semi automated  procedure  was  established.  For  each  semi¬ 
monthly  SDI  run,  the  TAB  index  of  starred  uescriptors  was  matched  clerically 
against  a  master  alphabetical  list  of  all  descriptors  appearing  in  all  pro¬ 
files.  A  punch  card  was  prepared  for  each  hit,  and  these  cards  were 
processed  to  provide  a  list  of  hits,  sorted  in  sequence  with  their  appearance 
in  the  TAB.  Each  hit  citation  was  cut  from  the  TAB  and  marked  to  indicate 
the  code  numbers  for  all  subscribers  to  receive  that  citation.  A  proper 
number  of  copies  were  made  of  each  hit  citation,  and  the  reproductions  were 
sorted  into  packets  for  each  subscriber.  Each  packet  was  covered  with  a 
preaddressed  order  sheet  to  form  a  compact  booklet  of  all  hit  citations  for 
each  subscriber.  A  sample  booklet  appears  in  Fig.  1. 

Evaluation 


During  the  18-month  Phase  I  test,  document  ordering  was  heavy  and  con¬ 
stant,  and  subscribers  voiced  great  satisfaction  with  the  service. 

The  choice  of  DDC  as  the  document  base  for  the  system  was  advantageous 
from  another  point  of  views  DDC  furnished  copies  of  all  documents  announced 
in  the  TAB  promptly  and  without  cost.  When  a  subscriber  ordered  an  announced 
document,  a  hard  copy  was  provided,  usually  in  about  10  days. 

Below  are  the  average  volume  figures  compiled  from  data  during  the 
entire  18-month  period. 


TABLE  I 

TYPICAL  SEMIMONTHLY  RUN  IN  THE  PHASE  I  PILOT  TEST 


Average 
Number  of 
Subscribers 

Average 
Number  of 
Referrals 

Per  Booklet  • 

Average 

Number  Ordered 
From  Each 
Booklet 

Percent 

Ordered 

35.0 

25.5. 

3.0 

11.7 

These  figures  indicate  the  heavy  use  that  was  made  of  the  service.  The 
data  are  reinforced  by  the  users*  evaluation  of  the  hard  copies  of  ordered 

documents  after  receipt. 


Based  on  a  six-month  sample,  29.5$  of  the  ordered  documents  were  evalu¬ 
ated  as  "of  extreme  interest,"  58$  as  "of  interest,"  and  12.5$  as  "of  no 
interest,"  indicating  cleatly  that  scientists  could  effectively  select  docu¬ 
ments  of  value  from  their  booklets.  In  reply  to  a  questionnaire,  almost  all 


Atta 

the  5 
the  ^ 


Selected  Abstracts"  Booklet  (Phase  I  System) 


subscribers  rated  the  system  highly,  and  asked  that  it  be  continued.  The  aver¬ 
age  relevance  of  citations  reported  by  the  subscribers  was  44$.  (The  lower 
ordering  figure  is  explained  by  the  fact  that  subscribers  usually  ordered  only 
those  relevant  citations  of  immediate  interest,  others  were  merely  noted  or 
filed,  or  in  many  cases  the  abstract  sufficed  to  brief  the  subscriber  on  the 
report. ) 

A  system  modeled  on  the  Phase  I  pilot  test  could  well  be  adopted  by  a 
small  laboratory  with  less  than  100  potential  subscribers.  However,  to  serve 
a  large  scientific  community  such  as  ECOM  with  2,700  technical  personnel,  more 
complete  automation  was  required. 

PHASE  II  SYSTEM  (FULLY  COMPUTERIZED  SERVICE) 

Encouraged  by  the  reaction  to  the  Phase  I  test,  it  was  decided  to  fully 
automate  the  system  on  ECOM's  Burroughs- 5 500  computer,  using  as  the  document 
input  the  computer  tape  used  by  DDC  to  produce  the  TAB.  This  would  eliminate 
almost  all  the  clerical  chores  involved  in  preparing  and  distributing  the 
Selected  Abstract  notices  and  would  enable  larger  subscribership .  Approval 
to  inaugurate  Phase  II  was  granted  by  ECOM  headquarters  in  November  1965. 

After  being  briefed  on  the  proposed  system,  DDC  agreed  to  supply  the  required 
computer  tapes  on  a  regular  basis. 

Computer  Program 

A  COBOL  computer  program  was  prepared  by  the  ECOM  Computation  Agency, 
and  the  system  became  fully  operable  in  September  1966.  A  simplified  system 
flow  chart  (Fig.  2)  shows  the  major  inputs  and  outputs.  The  inputs  are  (1) 
the  semimonthly  DDC  tape  containing  about  2,000  new  citations  indexed  and 
abstracted;  (2)  a  tape  containing  all  subscriber  profiles ;  (3)  punch  cards 
for  updating  the  profile  file;  and  (4)  a  tape  containing  all  keywords  in  the 
DDC  thesaurus  to  validate  the  profile  file.  Goaqplet®  flow  charts  for  the 
system  appear  in  the  Appendix,  along  with  run  descriptions. 

The  Phase  II  system  performs  automatically  all  the  clerical  procedures 
of  the  Phase  I  system  even  to  printing  the  Selected  Abstract  booklet  for  each 
subscriber;  however,  bursting,  stapling  and  mailing  are  clerical  functions. 

The  computer  also  prepares  the  pre addressed  cover  sheet  used  to  route  the  book¬ 
let  to  the  user  and,  in  turn,  to  serve  as  a  document  order  form.  A  sample 
cover  sheet  and  an  abstract  printout  appear  in  Figures  3  and  4. 

Also  produced  are  a  series  of  statistical  reports  for  system  evaluations 
one  listing  all  the  hits  for  each  subscriber  by  keyword  (Fig.  5) ;  one  giving 
the  number  of  documents  referred  to  each  subscriber  and  a  cumulative  tally 
(Fig.  6) ;  one  listing  the  hits  alphabe ti cally  by  keyword  (Fig.  7);  one  listing 
all  current  profiles  (Fig.  8) ;  and  one  listing  the  users  for  each  keyword  in 
the  profile  file  (Fig.  9). 

Subscribership 

3 

In  preparing  for  subscribership  increase,  the  SDI  Profiling  Handbook 
was  printed  in  large  numbers ,  and  a  promotional  brochure  was  issued.  Subscrib¬ 
ership  was  boosted  in  stages  from  50  to  100,  to  250,  to  400,  and  presently 
stands  at  520.  Of  these  subscribers,  about  100  are  from  military  agencies 
outside  of  ECOM,  who  requested  enrollment  on  a  test  basis  after  becoming  aware 
of  the  system  either  in  the  New  York  Times, ^  or  in  the  trade  magazines. 

5 


PROFILE 

CHANGES 


THESAURUS 


PROFILES 


ANALYSIS 

REPORTS 


Fig,  2.  Simplified  System  Flow  Chart  (Phase  II  System) 
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Among  these  ares  Edgevood  Arsenal,  Aberdeen  Proving  Ground,  Fort  Detrick, 
Fort  Totten,  West  Point,  the  Corps  of  Engineers,  Sandia  Air  Force  Base,  and 
the  Joint  Chiefs  of  Staff.  These  enrollees  were  accepted  to  test  the 
applicability  of  the  -system  to  external  agencies  in  preparation  for  the  Phase 
III  program. 

The  target  ECOM  enrollment  for  FY-69  is  1,000,  which  is  considered  the 
saturation  point  for  ECOM  with  its  2,700  technical  personnel.  Many  sub¬ 
scribers  who  have  common  interests  share  their  Selected  Abstract  notices  with 
co-workers . 


The  subscriber  dropout  rate  has  been  extreme 
five-year  period,  only  4.0  users  discontinued  the 
due  to  retirement  or  .job  changes.  Twelve  of  the 
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Enrollees  consist  mostly  of  scientists 
also  include  managers ,  military  personnel ,  : 
systems  analysts.  Subject  coverage  is  bread 
in  the  SDI  Profiling  Handbook,  2,843  were  s> 


,  engineers,  and  technicians 
instructors,  project  leaders 
*  Cf  the  7,145  descriptors 
elected  as  profile  terms  by 


,  but 
,  and 
lists! 
ECOM 


subscribers. 


Profiling 


AH  new  subscribers  during  the  Phase  II  system  prepared  their  own  pro¬ 
files  by  referring  to  the  User  Profiling  Handbook , ^  A  profile  analyst  was 
available  for  assistance,  but  was  rarely  called  upon.  His  primary  functio 


was  to  compare  the  profile  keywords  submitted,  by 
narrative  statement  of  the  user’s  interests  and 
seemed  required.  Ha  also  reviewed  in  detail  wit 
that  retrieved  an  excessive  number  of  citations. 


■  the  subscriber  with  the 
rn commend  changes  where  the' 
•h  the  subscriber  profiles 
cr  profiles  that  ware  pro- 
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ducing  many  irrelevant  citations.  To  correct  these  citations,  the  analyst 
could'"  use  the  improved  capability  provided  by  the  Phase  II  system  to  employ 
"negative  keywords”  in  the  profile  to  reduce  unwanted  citations. 


In  general ,  subscr: 
terms  for  their  orof 13 « 
compiled  from  keyword  6. 
both  alphabetically  and 
subscriber  takes  about 


.bers  had  little  difficulty  locating  the  appropriate 
from  the  profiling  handbook ,  which  was  specially 


ita  provided  by  DDC.  The  handbook  lists  keywords 
by  C0SATI*  subject  fields  and  groups .  A  typical 


zc  como  1  &  *C€ 


rot: 


age  consists  of  15  terms. 


Selection 


Belov  is  a  sample  document  citation 
is  coded  onto  "the  computer  tape  supplied 
in  the  ECOM  3,01  system.* 


m  it  appears  in  the  TAB*  and  as  it 
;;eui>-uhu:ly  oy  DDC  for  processing 


*  0  omini  t  te  e  o  n  3  c  .1  e  n ti  f  i  c  and 
Council  on  Science  and  Teehnolorru 


mica  i 


Information  of  the  Federal 


AD-653  060  Fid.  20/5 
CFSTI  Wcess  HC  $3.00  MF  $0.65 
MINNESOTA  UN IV  MINNEAPOLIS  DEPT 
OF  ELECTRICAL  ENGINEERING 
ACTIVE  DISTORTION  DURING  RUBY  LASER 
ACTION. 

Technical  rept., 

by  Mvlan  Radulovich  and  R.  J.  Collins.  Apr  67, 

58 p.  Rept.  no.  TR-5 

Contract  Noqr-7 10  (61 ),  Proj.  NR-015-704 

Unclassified  report 

Descriptors:  (*  Lasers,  Distortion),  (*Ruby, 

Thermal  stresses).  Pumping  (Optical).  Ther¬ 
mal  properties.  Optical  properties,  Thermal 
conductivity.  Interferometers,  Heating,  Ther¬ 
mal  expansion.  Refractive  index 

Ruby  laser  rods  were  studied  with  respect  to  ther¬ 
mal  distortions  induced  by  the  excitation  pump 
flash.  Nonuniform  temperature  distributions  cause 
optical  distortions  and  prevent  the  laser  beam  from 
realizing  its  minimum  theoretical  width  (diffraction 
limit).  These  variations  result  from  the  radial  distri¬ 
bution  of  absorbed  energy.  The  radial  gradients 
produce  changes  in  the  index  of  refraction  and  op¬ 
tical  path  length  which  lead  to  a  lens  type  behavior 
by  the  ruby  rod.  More  uniform  pumping  would  re¬ 
duce  the  temperature  distributions  and  conse¬ 
quently  diminish  this  lens  effect.  (Author) 

Fig.  10  Sample  Citation 

Approximately  2,000  citations  of  this  type  appear  in  each  semimonthly 
tape.  In  addition  to  the  normal  bibliographic  data  and  abstract,  the  citation 
contains  a  cluster  of  descriptors  (keywords)  assigned  by  DDC  indexers.  Note 
also  that  two  of  the  12  terms  are  preceded  by  an  asterisk,  indicating  that 
these  are  topics  of  major  concern  in  the  document.  The  other  terms  act  as 
modifiers ,  or  indicate  concepts  of  lesser  importance  in  the  document. 

Since  both  the  keywords  assigned  to  each  document  and  the  keywords  in 
each  subscriber's  interest  profile  are  selected  from  the  same  thesaurus,  a 
definite  direct  match  between  keywords  on  the  two  files  is  possible. 

In  the  Phase  II  system,  all  subscribers  who  have  one  of  the  starred  terms 
(LASERS  or  RUBY)  as  one  of  their  profile  keywords  would  be  sent  the  above 
citation,  except  if  their  profile  contained  as  a  negatively  coded  term  one  of 
the  other  terms  assigned  to  the  document  (for  instance  REFRACTIVE  INDEX) . 

A  sample  profile  that  would  retrieve  this  citation  is  shown  below: 
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The  introduction  of  negative  descriptors  in  tha  Phase  II  system  improved 
the  selectivity  of  the  matching  strategy  used  in  Phase  I.  For  example,  a  sub¬ 
scriber  who  has  the  term  PROPAGATION  as  a  keyword  but  is  not  interested  in  the 
propagation  of  mechanical  waves  could  add  the  negative  descriptors  ACOUSTICS 
and.  SOUND  to  eliminate  the  unwanted  citations® 

Evaluation  -  -?  - 

1 .  Document  Ordering 

Detailed  statistics  were  recorded  on  the  number  of  documents  referred 
to  each  subscriber  and  the  corresponding  number  ordered.  Summary  data  are 
presented  for  the  moat  recent  15  runs  (see  Tables  II  and  III). 

The  heavy  ordering  volume  is  an  indicator  of  system  acceptance  and 
utility.  Mote  that  on  the  average,  12.5$  of  the  citations  sent  to  subscribers 
were  ordered.  (The  overall  range  was  9.5$  to  20.9$)  These  figures  are  typi¬ 
cal  ©f  the  ordering  response  during  the  entire  five-year  period.  Mote  also 
that  starred  columns  in  Table  II  tend  to  have  higher  ordering  percentages 
because  the  DBG  tapes  for  those  runs  contained  only  unclassified-unlimited 
citations.  When  classified  citations  are  included,  the  percentage  drops 
several  points,  probably  because  of  difficulties  in  handling  and  storage. 

An  analysis  of  the  ordering  habits  of  subscribers  for  five  runs  indi¬ 
cates  s 


(a)  are  steady  users,  i.e. ,  documents  were  ordered  from  all  five 

runs  or  fro®  four  out  of  five  runs. 

(b)  17$  are  moderate  users ;  i.e.,  documents  were  ordered  from  three 

out  of  five  runs® 

(e)  34$  are  light  users?  i.e.,  documents  were  ordered  from  only  one 

or  two  of  five  runs® 

(d)  11$  did  not  order  at  all.  They  were  contacted  for  possible 

profile  modification. 

During  these  five  runs,  56$  of  the  Selected  Abstract  booklets  re¬ 
sulted  in  at  least  on©  order.  This  performance  is  considered  satisfactory, 
although  it  could  be  considerably  improved  with  a  more  active  program  to 

troubleshoot  profiles  of  subscribers  who  order  infrequently. 

2®  Questionnaire 

It  was  evident  that  the  actual  relevance  of  citations  was  higher  than 
the  12.5$  ordering  figure.  Documents  of  interest  wear©  not  ordered  for  in¬ 
stance,  if  the  document  had  already  been  seen,  if  the  subscriber  was  already 
familiar  with  the  work,  if  the  abstract  sufficed  for  the  present,  if  the 
subscriber's  reading  load  was  great,  or  if  the  document  did  not  have  an  imme¬ 
diate  bearing  on  his  work. 

To  get  a  better  picture  of  subscriber  reaction  to  the  service,  a 
questionnaire  was  sent  out  along  with  the  Selected  Abstract  booklets  to  118 
subscribers  who  had  been  enrolled  for  at  least  four  runs.  Ninety-six 
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questionnaires  were  returned  completed.  Summary  data  on  responses  to  signifi¬ 
cant  questions  appear  below. 

Q:  Are  you  receiving  too  many  citations  in  your  Selected  Abstract  notices? 

As  Yes  11$;  No  89$. 

Q:  What  percent  of  the  citations  in  the  attached  printout  is  pertinent  to 
your  interests? 

A:  Average  of  figures  stated  was  39.7$. 

Q:  Is  this  a  typical  percentage? 

A:  Yes  68$;  No  22$;  No  answer  10$. 

Qs  If  not,  please  estimate  typical  percentage  of  pertinent  documents  on  an 
overall  basis. 

A:  Some  stated  higher  percentages,  some  lower  percentages  than  in  previous 
question,  producing  a  corrected  relevance  figure  of  40.2$. 

Qs  In  general ,  do  you  think  the  percent  of  relevant  documents  is  satisfac¬ 
tory? 

A:  Yes  92$,  No  5$;  No  answer  3$. 

Q;  Are  you  receiving  ordered  documents  promptly? 

A:  Yes  95$;  No  2$;  No  answer  3$. 

Q:  Are  reproductions  satisfactorily  legible? 

As  Yes  95$;  No  4$;  No  answer  1$. 

Qs  Please  indicate  below  the  effect  this  service  has  had  on  your  work. 

Check  appropriate  items. 

A:  89$  Revealed  valuable  documents  that  probably  would  not  have  been  dis¬ 
covered  by  subscriber. 

69$  Saved  time  in  literature  searching. 

34$  Revealed  other  persons  working  in  area  who  later  provided  information 
of  value. 

28$  Affected  technical  decisions  in  work  area. 

27$  Increased  productivity. 

20$  Revealed  duplication  of  effort. 

20$  Shortened  time  required  to  complete  a  particular  task. 

14$  Indicated  some  anticipated  work  was  unnecessary. 

4$  Changed  course  of  work. 

3$  No  benefit. 

0$  Affected  choice  of  contractor. 

Qs  What  is  your  general  reaction  to  the  service? 

As  Narrative  comments  were  rated  as  follows s 

Excellent  49$;  Good  33$;  Moderate  7$;  Unfavorable  1$;  No  comment  10$. 

Qs  Do  you  wish  to  be  dropped  as  a  subscriber? 

A:  Yes  3$;  No  97$. 

Two  items  are  of  particular  significance  s  the  answers  in  regard  to  citation 
relevance  and  to  effect  on  the  individual ' s  work. 

3.  Relevance.  Since  in  the  EC0M  system  subscribers  are  not  required  to 
feed  back  relevance  evaluations  for  each  citation,  an  indication  of  overall 
relevance  was  sought  in  the  first  five  questions.  Relevance  is  about  40$, 
and  subscribers  are  generally  satisfied  with  this  relevance. 
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Document  Ordering  for  Latest  15  Runs 


1*  2*  3# 


12  13  Ht  15  MEAN 


NUMBER  OF 

SUBSCRIBERS  119  119  125  135  212  212  212  385  386  386  390  1*00  1*10  1*55  1*73 


DOCUMENTS 

REFERRED 


DOCUMENTS 

ORDERED 


PERCENT 

ORDERED 


988  ll*6U  1321*  2951  1^910  3751i  5258  3392  U392  3999  U6U1  901*5  81*62  91*1*2  1*8 


8  268 


561*  675 


SB 


63  879  1060  6 


18.0  18.3  20.9  Hi.6  11.2  11*. 5  12.9  9.5  ll*.l  ll*.5  12.2  10.9  10.1*  11.2  12.5 


4.  Effect  __on  Work.  The  responses  to  question  8  clearly  indicate  that 

the  service  is  having  a  significant  impact  on  the  R&D  program  in  terms  of 
awareness  of  duplication,  shortened  time  required  to  complete  a  particular  task, 
reduced  time  in  literature  searching,  effect  on  technical  decisions,  and  ^ 

altering  of  course  of  work.  The  percentages  for  many  of  the  items  were  much 
higher  than  expected. 

Those  who  checked  the  item  "Saved  time  in  literature  searching"  were 
asked  to  estimate  the  number  of  hours  saved.  Of  those  who  checked  this  item, 
the  average  figure  given  was  a  significant  3.4  hours  per  week. 

It  would  be  valuable  for  other  SDI  systems  to  obtain  additional  data 
in  the  area  of  work  impact.  It  is  expected  that  the  results  will  be  similar. 
Such  results  present  a  powerful  argument  to  management  for  adoption  or  ex¬ 
pansion  of  SDI  services  in  an  R&D  community,  probably  more  so  than  relevance 
figures. 

5.  Operating  Costs 

The  total  cost  for  operating  the  system,  based  on  the  cost  for  one 
typical  run  with  500  subscribers,  is  $1,229  per  run,  or  $2.45  per  subscriber. 
Translated  to  annual  figures,  the  system  costs  $29,496  per  year,  or  about  $58 
per  subscriber. 

The  largest  cost  item  is  $934  per  run  for  computer  processing.  This 
estimate  is  based  on  operation  on  a  Burroughs-5500  computer  owned  by  ECOM, 
with  costs  prorated  to  the  computer  run  of  many  jobs  simultaneously. 

The  amount  of  actual  computer  process  time  was  4.4  hours,  with  an  addi¬ 
tional  5.9  hours  for  printing  Selected  Abstract  notices  from  backup  tapes. 


The  figures  break  down  as  follows: 

Computer  Processing 

$727.00 

Backup  Printing 

177.00 

Keypunching 

30.00 

Paper  and  Supplies 

175.00 

Bursting  and  Stapling 

20.00 

Processing  Document  Orders 

50.00 

Profiling  and  Administration 

50.00 

Total 

$1,229.00 

The  low  operating  cost  of  the 

ECOM  SDI  system  is  attributable  to  the 

fact  that  some  of  the  high-cost  items  inherent  in  many  other  systems  are 
avoided,  primarily  because  of  ECOM's  use  of  DDC  as  a  document  data1  base.  As 
part  of  its  normal  operations,  DDC  performs  the  costly  functions  of  document 
acquisition,  document  storage ,  abstracting,  indexing,  and  keypunching  citations. 

PHASE  III  SYSTEM  (DEVELOPMENT  OF  STANDARD  ARMY  SYSTEM) 

In  September  1964,  ECOM  suggested  that  SDI  be  enrolled  in  the  planned 
National  ADP  Program  for  AMC  Logistics  Management  (NAPALM) .  This  Army 
Materiel  Command  (AMC)  program  is  an  effort  to  establish  a  bank  of  standard 
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computer  programs  in  the  logistics  area,  for  us©  by  all  AMG  installations  ,  and 
to  emplace  standard  compatible  computer  equipment  at  all  installations. 

EGOM  was  assigned  subproponency  for  SDI  under  NAPALM  in  February  1965, 
and  a  preliminary  specification  for  a  standard  SDI  system?  was  developed. 

This  specification  was  later  updated,  based  on  experience  with  the  operational 
Phase  II  system  at  BOOM. 

In  December  1967,  a  complete  reevaluation  of  the  Phase  II  system  restated 
in  development  of  a  new  detailed  program  specification.  The  computer  programs 
to  implement  this  specification  comprise  the  Phase  III  system  now  being  pro¬ 
grammed  and  debugged  prior  to  NAPALM  implementation  in  1969. 

Mai or  Improvements 

The  new  system  is  basically  modeled  on  the  Phase  II  system,  but  the 
operational  procedure  is  considerably  streamlined  by  combining  programs  and 
using  random  access  disk  storage.  In  addition,  the  following  new  features 
are  added: 

(1)  Full  boolean  strategy  has  been  incorporated  to  increase  relevance 
and  permit  processing  of  document  files  other  than  DDG’s®  Under  this  selec¬ 
tion  system,  a  subscriber  can  express  each  of  his  interest  areas  as  a  cluster  of 
logically  related  keywords.  By  appropriate  coding  of  his  keywords ,  for  in¬ 
stance,  he  can  say  in  effect,  "I  am  interested  in  all  documents  that  are 
indexed  with  either  term  A  or  term  B,  but  only  if  term  C  or  D  or  E  is  also 
present;  moreover ,  if  term  G  or  H  or  I  is  present,  I  am  not  interested  in  the 
document."  The  computer  will  then  select  for  that  subscriber  only  those  docu¬ 
ments  whose  keyword  clusters  have  the  appropriate  combination  of  keywords . 

(2)  The  computer  automatically  produces  for  each  subscriber  semi¬ 
annually  a  profile  performance  summary,  which  reports  on  the  activity  of  each 
keyword  during  the  period  so  that  the  subscriber  can  make  appropriate  changes. 

A  similar  report  is  automatically  produced  when  a  subscriber  fails  to  order 
documents,  or  if  his  profile  is  producing  an  excessive  number  of  citations. 

(3)  The  Selected  Abstracts  notice  has  been  re-formatted  for  increased 
readability.  Side-by-side  printing  of  two  abstracts  at  once  will  cut  print¬ 
ing  time  in  half  and  significantly  reduce  the  cost  of  paper. 

(4)  Automatic  security  controls  are  incorporated  to  notify  the  system 
immediately  when  the  security  status  of  a  subscriber  changes ,  and  to  prevent 
ordering  of  a  classified  document  for  an  unauthorized  individual. 

A  simplified  system  flow  chart  for  the  Phase  III  system  appears  in 
Figures  11a  and  11b. 

Automatic  Profile  Review 

Item  (2)  above,  because  of  its  unique  approach,  deserves  special  mention. 
The  subscriber*  s  interest  profile  is  not  a  stable  thing.  Continual  modifica¬ 
tion  is  required  to  correct  errors  or  mis judgments  in  selecting  keywords ,  and 
to  reflect  changes  in  the  subscriber ’  s  interests  over  a  period  of  time.  Since 
subscribers  cannot  be  depended  upon  to  take  the  initiative  in  changing  their 
profiles ,  it  becomes  incumbent  upon  the  SDI  system  to  provide  for  systematic 
updating  of  profile  keywords . 
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Fig.  Ha.  Flow  Chart  for  Phase  III  System,  Part  I 
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The  Phase  III  computer-assisted  profile  updating  system  is  based  on  the 
assumption  that  the  subscriber  himself  is  in  the  best  position  to  refine  his 
profile,  and  can  perform  the  task  effectively  if  provided  with  the  proper  data. 
Direct  modification  of  profiles  by  users  has  the  additional  psychological  ad¬ 
vantage  of  letting  the  subscriber  feel  the  system  is  directly  responsive  to 
his  will.  To  this  end,  the  system  generates  semiannually  a  profile  report  to  each 
subscriber.  It  lists  his  profile  terras,  each  one  annotated  with  the  number  of 
documents  referred  and  ordered  under  that  keyword. 

The  referred- to-ordered  ratio  for  each  term  gives  the  subscriber  a  pre¬ 
cise  indication  of  the  effectiveness  of  each  term  in  his  profile,  and  enables 
him  to  spot  unfruitful  or  erroneous  keywords.  The  report  is  designed  so  that 
changes  can  be  made  easily.  Penciled-in  changes  can  be  keypunched  directly 
from  the  form.  Full  instructions  on  how  to  make  changes  are  provided  with  the 
report,  and  each  report  is  accompanied  by  a  User  Profile  Handbook. 

These  profile  review  reports  are  automatically  generated  for  one  sixth  of 
the  subscribers  each  month,  thus  providing  for  semiannual  profile  evaluation 
for  all  subscribers  on  a  phased  basis.  In  addition,  reports  are  to  be  printed 
out  for  subscribers  who  have  not  ordered  any  document  for  three  months,  for 
those  whose  profile  is  retrieving  more  than  100  citations  a  month,  and  for 
those  who  have  changed  jobs. 

Microfiche  Ordering 

In  May  1968,  DDC  announced  that  effective  1  July  1968,  hard  copy  of 
announced  documents  would  no  longer  be  provided  free  of  cost,  but  at  a  charge 
of  three  dollars  per  copy;  however,  microfiche  would  be  furnished  free.  Since 
the  SDI  system  results  in  about  20,000  document  orders  a  year,  continued 
ordering  of  hard  copy  could  become  prohibitively  costly. 

The  microfiche  shown  in  Fig.  12  is  a  4-  by  5  3/4-inch  sheet  of  film  con¬ 
taining  60  report  pages  that  can  be  read  on  a  special  viewer  to  enlarge  the 
page  image.  To  encourage  the  use  of  microfiche,  EC0M  has  purchased  micro¬ 
fiche  reader-printers,  which  make  photocopy  of  a  projected  page  in  six  seconds , 
and  also  portable  readers  (without  the  printing  capability).  This  equipment 
is  placed  strategically  throughout  the  agency  for  use  of  SDI  subscribers. 

Present  plans  are  to  order  only  microfiche  for  SDI  users  on  the  initial 
order,  and  honor  orders  for  hard  copy  only  after  the  subscriber  has  read  the 
microfiche  and  determines  he  needs  a  hard  copy  of  the  document.  The  Phase  III 
system  has  built-in  provisions  to  implement  this  plan. 

FUTURE  PLANS 

The  Phase  III  system  is  presently  being  programmed  for  the  Burroughs- 5 500 
or  IBM-360  NAPALM  computer,  and  should  be  fully  operational  in  January  1969. 

The  proposed  system  will  then  be  tested  and  coordinated  with  potential  user 
installations. 

One  of  the  future  requirements  envisioned  is  addition  of  pre-edit  pro¬ 
grams  so  that  files  other  than  DDC's  can  be  input  to  the  system.  Some  of  the 
abstract  files  available  on  tape  include  those  produced  by  Chemical  Abstracts, 
Index  Medicus,  Engineering  Index,  the  Institute  for  Scientific  Information, 
and  Pandex.  The  Phase  III  system  has  been  designed  so  that  it  will  be  a 
relatively  simple  matter  to  process  these  other  inputs,  if  required. 
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CONCLUSIONS  AND  RECOMMENDATIONS 

The  SDI  system  described  has  proved  notably  effective  in  serving  the 
information  needs  of  ECOM  and  other  Army  technical  personnel ,  by  notifying 
them  regularly  of  new  documents  published  in  their  fields  of  interest. 
Response  of  subscribers  has  been  enthusiastic  as  reflected  in  the  results  of 
questionnaires ,  ordering  volume ,  and  steady  growth  of  enrollment. 

The  Phase  I  semiautomated  test  established  the  desirability  of  such  a 
service  at  ECOM.  The  Phase  II  pilot  operation,  besides  showing  that  the 
service  can  be  effectively  and  economically  implemented  by  computer,  formed 
the  basis  for  development  of  the  improved  Phase  III  system. 

Use  of  the  preprocessed  current  accessions  tape  of  DDC  as  the  document 
base  for  an  SDI  system  offers  advantages  of  breadth  of  coverage ,  low  key¬ 
punching  load,  availability  of  announced  documents,  and  ease  of  effective 
keyword  matching. 

The  system  has  had  a  significant  impact  on  the  RDTE  program  at  ECOM  in 
terms  of  reduced  duplication  of  effort,  reduced  time  spent  by  scientists  in 
library  searching,  redirection  of  technical  effort,  and  increased  productiv¬ 
ity. 


As  a  concrete  example  of  the  potential  for  cost  savings  implicit  in  SDI, 
a  subscriber  recently  reported  a  saving  of  $42,000  and  750  man-hours  as  a 
result  of  one  item  of  information  provided. 

It  is  recommended  that  military  R&D  activities  consider  the  establishment 
of  an  SDI  system,  such  as  the  one  described,  to  help  their  technical  personnel 
cope  with  the  information  explosion  and  more  effectively  apply  the  results  of 
external  scientific  work. 
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APPENDIX 


RON  DESCRIPTIONS  FOR  PHASE  II  PROGRAM 
ROUTINE  I;  TAPE  CONVERSION  (See  Fig.  A-1) 

These  programs  (STNCON  and  STN000)  reformat  the  semimonthly  DDC  document 
accession  tape  and  create  two  intermediate  tapes s  A02,  which  contains  all 
keywords  on  the  tape  and  their  associated  document  numbers,  and  A01 ,  which 
contains  the  full  document  citations. 

RPUTXHE  IT  •-  SET  .BOTTOM  OF  DOCUMENTS  FOR  EACH  USER  (See  Fig.  A-2) 

This  set  of  programs  (STN001  through  STN006)  makes  the  decision  on  which 
documents  are  to  be  referred  to  each  subscriber  and  prepares  an  addressed 
booklet  of  selected  citations  for  each  subscriber. 

STN001  sorts  the  document  keywords  into  alphabetical  order  for  batch 
processing. 

STN002  records  every  match  between  a  document  and  a  profile  keyword. 

STN003  selects  from  all  the  hits  only  those  that  matched  on  starred  docu¬ 
ment  keywords,  eliminates  duplicate  hits  for  a  subscriber,  and  rejects  cita¬ 
tions  which  hit  on  a  negative  descriptor. 

STN004  writes  out  a  full  document  citation  for  each  retrieval. 

STN005  sorts  these  retrievals  into  subscriber  number  order  in  preparation 
for  printing. 

STN006  prepares  an  addressed  cover  sheet  for  each  booklet,  and  prints 
out  the  full  citations,  including  abstract  if  present,  for  all  retrieved 
citations  for  each  subscriber. 

ROUTINE  III:  STATISTICAL  REPORTS  (See  Fig.  A-3) 

These  programs  (STN007  and  STN008)  prepare  three  reports  used  for 
troubleshooting  and  evaluation  of  system  effectiveness. 

STN007  provides  a  list  giving  the  number  of  retrievals  for  each  keyword 
in  each  subscriber's  profile  (See  Fig.  5),  and  a  list  giving  the  name  of  each 
subscriber,  the  number  of  documents  referred  per  cycle,  and  a  cumulative  total 
of  referrals  (See  Fig.  6) . 

ROUTINE  IVt  FITE  UPDATING  AND  VALIDATION  (See  Fig.  A-4) 

These  programs  (STN0AA,  STN0AB,  and  STN901)  accept  changes,  additions, 
and  deletions,  and  validate  all  profile  keywords  against  a  master  tape  of 
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authorized  keywords. 

STN0AA  updates  the  user  profiles  and  produces  a  printout  of  the  current 
profile  list  of  all  subscribers  (See  Fig.  7) . 

STM0AB  validates  the  entire  profile  keyword  file  by  comparing  each  key¬ 
word  with  an  authorized  list  of  all  DDC  descriptors,  and  produces  a  list  of 
all  subscribers  who  use  each  keyword  (See  Fig.  8)  and  a  punch  card  for  all 
invalid  descriptors  for  correction  in  the  next  cycle. 

STN9$1  updates  the  master  authority  list  of  all  DDC  descriptors . 
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UREA-2  SELECTION  OF  CITATIONS 
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FIGURE  A -3  STATISTICAL  REPORTS 
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FIGURE  A-4  FILE  UPDATING  &  VALIDATION 
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